Single-Trial Classification of Multi-User P300-Based Brain-Computer Interface Using Riemannian Geometry

International audienceThe classification of electroencephalographic (EEG) data recorded from multiple users simultaneously is an important challenge in the field of Brain-Computer Interface (BCI). In this paper we compare different approaches for classification of single-trials Event-Related Potential (ERP) on two subjects playing a collaborative BCI game. The minimum distance to mean (MDM) classifier in a Riemannian framework is extended to use the diversity of the inter-subjects spatio-temporal statistics (MDM-hyper) or to merge multiple classifiers (MDM-multi). We show that both these classifiers outperform significantly the mean performance of the two users and analogous classifiers based on the step-wise linear discriminant analysis. More importantly, the MDM-multi outperforms the performance of the best player within the pair

Mining the Bilinear Structure of Data with Approximate Joint Diagonalization

International audienceApproximate Joint Diagonalization of a matrix set can solve the linear Blind Source Separation problem. If the data possesses a bilinear structure, for example a spatio-temporal structure, transformations such as tensor decomposition can be applied. In this paper we show how the linear and bilinear joint diagonalization can be applied for extracting sources according to a composite model where some of the sources have a linear structure and other a bilinear structure. This is the case of Event Related Potentials (ERPs). The proposed model achieves higher performance in term of shape and robustness for the estimation of ERP sources in a Brain Computer Interface experiment

Approximate Joint Diagonalization within the Riemannian Geometry Framework

International audienceWe consider the approximate joint diagonalization problem (AJD) related to the well known blind source separation (BSS) problem within the Riemannian geometry framework. We define a new manifold named special polar manifold equivalent to the set of full rank matrices with a unit determinant of their Gram matrix. The Riemannian trust-region optimization algorithm allows us to define a new method to solve the AJD problem. This method is compared to previously published NoJOB and UWEDGE algorithms by means of simulations and shows comparable performances. This Riemannian optimization approach thus shows promising results. Since it is also very flexible, it can be easily extended to block AJD or joint BSS

"Brain Invaders 2" : an open source Plug & Play multi-user BCI videogame

International audienceIntroduction: In 2011 we proposed « Brain Invaders » [1], a BCI videogame inspired from the vintage game Space Invaders. The software was released open source and was compatible with OpenVIBE [2]. The system is based on ERP classification using the oddball paradigm with a grid of 36 possible targets. This second version extends the game to the multiuser scenario. It includes four game modes namely Solo, Collaboration, Cooperation, Competition which are suitable for hyperscanning studies. Thanks to a classification algorithm based on Riemannian geometry, the system shows very good accuracy and is fully " Plug & Play " , no calibration phase is needed

Brain Invaders Cooperative versus Competitive: Multi-User P300- based Brain-Computer Interface Dataset (bi2015b)

We describe the experimental procedures for a dataset that we have made publicly available at https://doi.org/10.5281/zenodo.3267307 in mat and csv formats. This dataset contains electroencephalographic (EEG) recordings of 44 subjects playing in pair to the multi-user version of a visual P300 Brain-Computer Interface (BCI) named Brain Invaders. The interface uses the oddball paradigm on a grid of 36 symbols (1 or 2 Target, 35 or 34 Non-Target) that are flashed pseudo-randomly to elicit the P300 response. EEG data were recorded using 32 active wet electrodes per subjects (total: 64 electrodes) during four randomised conditions (Cooperation 1-Target, Cooperation 2-Targets, Competition 1-Target, Competition 2-Targets). The experiment took place at GIPSA-lab, Grenoble, France, in 2015. Python code for manipulating the data is available at https://github.com/plcrodrigues/py.BI.EEG.2015b-GIPSA. The id of this dataset is bi2015b.Dans ce document, nous décrivons une expérimentation dont les données ont été publiées sur https://doi.org/10.5281/zenodo.3267307 aux formats mat et csv. Ce jeu de donnée contient les enregistrements électroencéphalographiques (EEG) de 44 sujets jouant par paires à une version multi-utilisateurs du jeu Brain Invaders (Congedo et al., 2011), une interface cerveau-ordinateur de type ‘P300 visuel’. L’interface repose sur le paradigme oddball avec une grille de 36 symboles (1 ou 2 Targets, 35 ou 34 Non-Targets) qui clignotent de façon pseudo-aléatoire afin de produire un P300, un potentiel évoqué apparaissant environ 300 ms après le début d’une stimulation. L'EEG de chaque sujet a été enregistré grâce à 32 électrodes humides réparties sur le scalp (total : 64 électrodes par pair), au cours de quatre sessions expérimentales randomisées (Cooperation 1-Target, Cooperation 2-Targets, Competition 1-Target, Competition 2-Targets). L'expérience a été menée au GIPSA-lab (Université de Grenoble-Alpes, CNRS, Grenoble-INP) en 2015. Nous fournissons également une implémentation python pour manipuler les données disponibles https://github.com/plcrodrigues/py.BI.EEG.2015b-GIPSA. L’identifiant de cette base de données est bi2015b

Methods for multi-subject electroencephalography and application to brain-computer interfaces

L'étude par neuro-imagerie de l'activité de plusieurs cerveaux en interaction (hyperscanning) permet d'étendre notre compréhension des neurosciences sociales. Nous proposons un cadre pour l'hyperscanning utilisant les interfaces cerveau-ordinateur multi-utilisateur qui inclut différents paradigmes sociaux tels que la coopération ou la compétition. Les travaux de cette thèse comportent trois contributions interdépendantes. Notre première contribution est le développement d'une plateforme expérimentale sous la forme d'un jeu vidéo multijoueur, nommé Brain Invaders 2, contrôlé par la classification de potentiels évoqués visuels enregistrés par électroencéphalographie (EEG). Cette plateforme est validée par deux protocoles expérimentaux comprenant dix-neuf et vingt-deux paires de sujets et utilise différentes approches de classification adaptative par géométrie riemannienne. Ces approches sont théoriquement et expérimentalement comparées et nous montrons la supériorité de la fusion des classifieurs indépendants sur la classification d'un hypercerveau durant la seconde contribution. L'analyse de coïncidence des signaux entre les individus est une approche classique pour l'hyperscanning, elle est pourtant difficile quand les signaux EEG concernés sont transitoires avec une grande variabilité (intra- et inter-sujet) spatio-temporelle et avec un faible rapport signal-à-bruit. En troisième contribution, nous proposons un nouveau modèle composite de séparation aveugle de sources physiologiquement plausibles permettant de compenser cette variabilité. Une solution par diagonalisation conjointe approchée est proposée avec une implémentation d'un algorithme de type Jacobi. A partir des données de Brain Invaders 2, nous montrons que cette solution permet d'extraire simultanément des sources d'artéfacts, des sources d'EEG évoquées et des sources d'EEG continues avec plus de robustesse et de précision que les modèles existants.The study of several brains interacting (hyperscanning) with neuroimagery allows to extend our understanding of social neurosciences. We propose a framework for hyperscanning using multi-user Brain-Computer Interfaces (BCI) that includes several social paradigms such as cooperation or competition. This dissertation includes three interdependent contribution. The first contribution is the development of an experimental platform consisting of a multi-player video game, namely Brain Invaders 2, controlled by classification of visual event related potentials (ERP) recorded by electroencephalography (EEG). The plateform is validated through two experimental protocols including nineteen and twenty two pairs of subjects while using different adaptive classification approaches using Riemannian geometry. Those approaches are theoretically and experimentally compared during the second contribution ; we demonstrates the superiority in term of accuracy of merging independent classifications over the classification of the hyperbrain during the second contribution. Analysis of inter-brain synchronizations is a common approach for hyperscanning, however it is challenging for transient EEG waves with an great spatio-temporal variability (intra- and inter-subject) and with low signal-to-noise ratio such as ERP. Therefore, as third contribution, we propose a new blind source separation model, namely composite model, to extract simultaneously evoked EEG sources and ongoing EEG sources that allows to compensate this variability. A solution using approximate joint diagonalization is given and implemented with a fast Jacobi-like algorithm. We demonstrate on Brain Invaders 2 data that our solution extracts simultaneously evoked and ongoing EEG sources and performs better in term of accuracy and robustness compared to the existing models

Méthodes pour l'électroencéphalographie multi-sujet et application aux interfaces cerveau-ordinateur

The study of several brains interacting (hyperscanning) with neuroimagery allows to extend our understanding of social neurosciences. We propose a framework for hyperscanning using multi-user Brain-Computer Interfaces (BCI) that includes several social paradigms such as cooperation or competition. This dissertation includes three interdependent contribution. The first contribution is the development of an experimental platform consisting of a multi-player video game, namely Brain Invaders 2, controlled by classification of visual event related potentials (ERP) recorded by electroencephalography (EEG). The plateform is validated through two experimental protocols including nineteen and twenty two pairs of subjects while using different adaptive classification approaches using Riemannian geometry. Those approaches are theoretically and experimentally compared during the second contribution ; we demonstrates the superiority in term of accuracy of merging independent classifications over the classification of the hyperbrain during the second contribution. Analysis of inter-brain synchronizations is a common approach for hyperscanning, however it is challenging for transient EEG waves with an great spatio-temporal variability (intra- and inter-subject) and with low signal-to-noise ratio such as ERP. Therefore, as third contribution, we propose a new blind source separation model, namely composite model, to extract simultaneously evoked EEG sources and ongoing EEG sources that allows to compensate this variability. A solution using approximate joint diagonalization is given and implemented with a fast Jacobi-like algorithm. We demonstrate on Brain Invaders 2 data that our solution extracts simultaneously evoked and ongoing EEG sources and performs better in term of accuracy and robustness compared to the existing models.L'étude par neuro-imagerie de l'activité de plusieurs cerveaux en interaction (hyperscanning) permet d'étendre notre compréhension des neurosciences sociales. Nous proposons un cadre pour l'hyperscanning utilisant les interfaces cerveau-ordinateur multi-utilisateur qui inclut différents paradigmes sociaux tels que la coopération ou la compétition. Les travaux de cette thèse comportent trois contributions interdépendantes. Notre première contribution est le développement d'une plateforme expérimentale sous la forme d'un jeu vidéo multijoueur, nommé Brain Invaders 2, contrôlé par la classification de potentiels évoqués visuels enregistrés par électroencéphalographie (EEG). Cette plateforme est validée par deux protocoles expérimentaux comprenant dix-neuf et vingt-deux paires de sujets et utilise différentes approches de classification adaptative par géométrie riemannienne. Ces approches sont théoriquement et expérimentalement comparées et nous montrons la supériorité de la fusion des classifieurs indépendants sur la classification d'un hypercerveau durant la seconde contribution. L'analyse de coïncidence des signaux entre les individus est une approche classique pour l'hyperscanning, elle est pourtant difficile quand les signaux EEG concernés sont transitoires avec une grande variabilité (intra- et inter-sujet) spatio-temporelle et avec un faible rapport signal-à-bruit. En troisième contribution, nous proposons un nouveau modèle composite de séparation aveugle de sources physiologiquement plausibles permettant de compenser cette variabilité. Une solution par diagonalisation conjointe approchée est proposée avec une implémentation d'un algorithme de type Jacobi. A partir des données de Brain Invaders 2, nous montrons que cette solution permet d'extraire simultanément des sources d'artéfacts, des sources d'EEG évoquées et des sources d'EEG continues avec plus de robustesse et de précision que les modèles existants

Méthodes pour l'électroencéphalographie multi-sujet et application aux interfaces cerveau-ordinateur

The study of several brains interacting (hyperscanning) with neuroimagery allows to extend our understanding of social neurosciences. We propose a framework for hyperscanning using multi-user Brain-Computer Interfaces (BCI) that includes several social paradigms such as cooperation or competition. This dissertation includes three interdependent contribution. The first contribution is the development of an experimental platform consisting of a multi-player video game, namely Brain Invaders 2, controlled by classification of visual event related potentials (ERP) recorded by electroencephalography (EEG). The plateform is validated through two experimental protocols including nineteen and twenty two pairs of subjects while using different adaptive classification approaches using Riemannian geometry. Those approaches are theoretically and experimentally compared during the second contribution ; we demonstrates the superiority in term of accuracy of merging independent classifications over the classification of the hyperbrain during the second contribution. Analysis of inter-brain synchronizations is a common approach for hyperscanning, however it is challenging for transient EEG waves with an great spatio-temporal variability (intra- and inter-subject) and with low signal-to-noise ratio such as ERP. Therefore, as third contribution, we propose a new blind source separation model, namely composite model, to extract simultaneously evoked EEG sources and ongoing EEG sources that allows to compensate this variability. A solution using approximate joint diagonalization is given and implemented with a fast Jacobi-like algorithm. We demonstrate on Brain Invaders 2 data that our solution extracts simultaneously evoked and ongoing EEG sources and performs better in term of accuracy and robustness compared to the existing models.L'étude par neuro-imagerie de l'activité de plusieurs cerveaux en interaction (hyperscanning) permet d'étendre notre compréhension des neurosciences sociales. Nous proposons un cadre pour l'hyperscanning utilisant les interfaces cerveau-ordinateur multi-utilisateur qui inclut différents paradigmes sociaux tels que la coopération ou la compétition. Les travaux de cette thèse comportent trois contributions interdépendantes. Notre première contribution est le développement d'une plateforme expérimentale sous la forme d'un jeu vidéo multijoueur, nommé Brain Invaders 2, contrôlé par la classification de potentiels évoqués visuels enregistrés par électroencéphalographie (EEG). Cette plateforme est validée par deux protocoles expérimentaux comprenant dix-neuf et vingt-deux paires de sujets et utilise différentes approches de classification adaptative par géométrie riemannienne. Ces approches sont théoriquement et expérimentalement comparées et nous montrons la supériorité de la fusion des classifieurs indépendants sur la classification d'un hypercerveau durant la seconde contribution. L'analyse de coïncidence des signaux entre les individus est une approche classique pour l'hyperscanning, elle est pourtant difficile quand les signaux EEG concernés sont transitoires avec une grande variabilité (intra- et inter-sujet) spatio-temporelle et avec un faible rapport signal-à-bruit. En troisième contribution, nous proposons un nouveau modèle composite de séparation aveugle de sources physiologiquement plausibles permettant de compenser cette variabilité. Une solution par diagonalisation conjointe approchée est proposée avec une implémentation d'un algorithme de type Jacobi. A partir des données de Brain Invaders 2, nous montrons que cette solution permet d'extraire simultanément des sources d'artéfacts, des sources d'EEG évoquées et des sources d'EEG continues avec plus de robustesse et de précision que les modèles existants

REM Sleep Impairment May Underlie Sleep-Driven Modulations of Tinnitus in Sleep Intermittent Tinnitus Subjects: A Controlled Study

(1) Background: Poor sleep and fragmented sleep are associated with several chronic conditions. Tinnitus is an auditory symptom that often negatively combines with poor sleep and has been associated with sleep impairment and sleep apnea. The relationship between tinnitus psychoacoustic characteristics and sleep is still poorly explored, notably for a particular subgroup of patients, for whom the perceived loudness of their tinnitus is highly modulated by sleep. (2) Methods: For this observational prospective study, 30 subjects with tinnitus were recruited, including 15 “sleep intermittent tinnitus” subjects, who had reported significant modulations of tinnitus loudness related to night sleep and naps, and a control group of 15 subjects displaying constant non-sleep-modulated tinnitus. The control group had matching age, gender, self-reported hearing loss grade and tinnitus impact on quality of life with the study group. All patients underwent a polysomnography (PSG) assessment for one complete night and then were asked to fill in a case report form, as well as a report of tinnitus loudness before and after the PSG. (3) Results: “Sleep Intermittent tinnitus” subjects had less Stage 3 sleep (p p p p p < 0.05). These correlations were not present in the control group. (4) Conclusions: This study suggests that among the tinnitus population, patients displaying sleep-modulated tinnitus have deteriorated sleep quality. Furthermore, REM sleep characteristics may play a role in overnight tinnitus modulation. Potential pathophysiological explanations accounting for this observation are hypothesized and discussed

Spatio-temporal common pattern: A companion method for ERP analysis in the time domain

International audienceBackgroundAlready used at the incept of research on event-related potentials (ERP) over half a century ago, the arithmetic mean is still the benchmark for ERP estimation. Such estimation, however, requires a large number of sweeps and/or a careful rejection of artifacts affecting the electroencephalographic recording.New MethodIn this article we propose a method for estimating ERPs as they are naturally contaminated by biological and instrumental artifacts. The proposed estimator makes use of multivariate spatio-temporal filtering to increase the signal-to-noise ratio. This approach integrates a number of relevant advances in ERP data analysis, such as single-sweep adaptive estimation of amplitude and latency and the use of multivariate regression to account for ERP overlapping in time. ResultsWe illustrate the effectiveness of the proposed estimator analyzing a dataset comprising 24 subjects involving a visual odd-ball paradigm, without performing any artifact rejection. Comparison with Existing Method(s)As compared to the arithmetic average, a lower number of sweeps is needed. Furthermore, artifact rejection can be performed roughly using permissive automatic procedures. ConclusionThe proposed ensemble average estimator yields a reference companion to the arithmetic ensemble average estimation, suitable both in clinical and research settings. The method can be applied equally to event related fields (ERF) recorded by means of magnetoencephalography. In this article we describe all necessary methodological details to promote testing and comparison of this proposed method by peers. Furthermore, we release a MATLAB toolbox, a plug-in for the EEGLAB software suite and a stand-alone executable application